Lecture 27 - Your project. Deep Learning with H2O. P.2 Implement Deep Learning Model to ShinyApp
Deep Learning… This lecture is dedicated to the implementation of Deep Learning models into for our Data
Work overview (from previous lectures)
- re-arranging data as matrix - DONE
- fitting deep learning models - DONE
- testing the models - DONE
- saving models - DONE
- implementation in our ShinyApp… - to be covered in this lecture
In order to achieve this goal we will have to perform some steps:
- Learn how to use saved model in R to make predictions
- Develop and implement app usage strategy
- Return to our time series to identify time of anomaly
Using saved h2o model in R
In the prevous lecture we have saved our Deep Learning Model in the www folder:
#files with model
file.info("www/tmp/normality_model.bin/DeepLearning_model_R_1510597411656_1")
One of the Goals of this lecture will be to see how to use this model in R to make predictions
From: (R h2o load a saved model from disk in MOJO or POJO format)[https://stackoverflow.com/questions/45335697/r-h2o-load-a-saved-model-from-disk-in-mojo-or-pojo-format]
We know that two types of strategy are possible
- use a binary model
- export a model to MOJO (or POJO) form
Using Binary Model to make predictions
We will first use the first option to actually keep H2O on the background to make predictions
library(h2o)
Warning message:
In scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings, :
EOF within quoted string
library(tidyverse)
library(plotly)
Initializing H2O and loading the model
# initialize h2o
h2o.init()
H2O is not running yet, starting it now...
Note: In case of errors look at the following log files:
C:\Users\fxtrams\AppData\Local\Temp\RtmpwjnYF6/h2o_fxtrams_started_from_r.out
C:\Users\fxtrams\AppData\Local\Temp\RtmpwjnYF6/h2o_fxtrams_started_from_r.err
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
Starting H2O JVM and connecting: . Connection successful!
R is connected to the H2O cluster:
H2O cluster uptime: 2 seconds 104 milliseconds
H2O cluster version: 3.14.0.7
H2O cluster version age: 24 days
H2O cluster name: H2O_started_from_R_fxtrams_vyq853
H2O cluster total nodes: 1
H2O cluster total memory: 1.77 GB
H2O cluster total cores: 4
H2O cluster allowed cores: 4
H2O cluster healthy: TRUE
H2O Connection ip: localhost
H2O Connection port: 54321
H2O Connection proxy: NA
H2O Internal Security: FALSE
H2O API Extensions: Algos, AutoML, Core V3, Core V4
R Version: R version 3.2.5 (2016-04-14)
# load model
normality_model <- h2o.loadModel("www/tmp/normality_model.bin/DeepLearning_model_R_1510597411656_1")
We will also need to quickly construct the dataset we can predict the result
# ============= READ DATA =================
# Read our small data ...
DF_Data_Recent <- readRDS("DF_Data_Process_Recent.data")
DF_Equipm <- read_csv("DF_EquipmData.csv")
# data frame containing Event Names
DF_EvCode <- read_csv("DF_EvCodeDataProject.csv")
# Data manipulation and saving to the DF_TEMP
DF_TEMP <- DF_Data_Recent %>%
# join to decode equipment serial number
inner_join(DF_Equipm, by = "IDEquipment") %>%
# join to decode Event Code meaning
inner_join(DF_EvCode, by = "EventCode") %>%
# select only column needed
select(StartDate, Name, AnalogVal, EventText)
Will be using Machine #4 for that
# importing R function from file
source("to_matrix.R")
DF_M4 <- DF_TEMP %>%
to_matrix(filter_Event = "Tubing Process, resistance Ohm",
filter_Machine = "Machine #4",n_cols = 150)
Predicting anomaly on Machine 4
In order to predict on DF_M4 dataset we need to make it H2OFrame class. We need to load this object into H2O environment
# load dataset into H2O environment
test_M4 <- as.h2o(x = DF_M4, destination_frame = "test_M4")
|
| | 0%
|
|================================================================================================================| 100%
# plot the anomalies
h2o.anomaly(normality_model, test_M4) %>% as.data.frame() %>% plot.ts(type = "p")

This brings us to the point of create a ‘cutoff’ which I will set as 0.25. MSE values scored above this value will be identifyed as anomaly.
# save MSE values
mse_out <- h2o.anomaly(normality_model, test_M4) %>% as.data.frame()
# filter anomalies
rows_anomals <- which(mse_out > 0.25)
rows_normals <- which(mse_out <= 0.25)
Using these rows we can ‘label’ or subset our anomalous observations
# subset rows with matrices that shows the anomalies
matr_anomals <- DF_M4[rows_anomals, ]
# subset rows with matrices that does not show the anomalies
matr_normals <- DF_M4[rows_normals, ]
Just out of curiosity I will plot the anomalies values as 3D plot
plot_ly(z = matr_anomals, type = "surface")
This is wonderful! High anomalous spikes were detected!
And also those without
plot_ly(z = matr_normals, type = "surface")
There is no spikes!
Catching corresponding dates & time
Once we learned to process our data through the model, split ‘faulty’ observations we will need to come back to reference our faulty observations over time. Perhaps there are ways to solve this problem in a more efficient way, but I will attempt to:
- extract corresponding date time
- extract matrixes with faults for both values and time
- convert matrices to vectors
- join faulty observations with corresponding time, add new column with label “1”
- join ‘good’ observations with corresponding time, add new column with label “0”
- construct the dataframes and arrange them back
To extract the time… I have modified our function that extracts AnalogVal variable and saved that to the R script to_matrixDT.R
# importing R function from file
source("to_matrixDT.R")
Let’s use this function to extract corresponding Date and Time…
DF_M4DT <- to_matrixDT(DF_TEMP,
filter_Event = "Tubing Process, resistance Ohm",
filter_Machine = "Machine #4",
n_cols = 150)
now I can extract corresponding matrixes
# subset rows with matrices that shows the anomalies
matr_anomals <- DF_M4[rows_anomals, ]
matr_anomalT <- DF_M4DT[rows_anomals, ] #corresponding time date index
# subset rows with matrices that does not show the anomalies
matr_normals <- DF_M4[rows_normals, ]
matr_normalT <- DF_M4DT[rows_normals, ] #corresponding time date index
Convert to vectors, join and mutate
# For anomalies
DF_4V <- matr_anomals %>% t() %>% c() %>% as.data.frame()
colnames(DF_4V) <- "AnalogVal"
DF_4T <- matr_anomalT %>% t() %>% c() %>% as.POSIXct() %>% as.data.frame()
colnames(DF_4T) <- "StartDate"
DF_A <- DF_4T %>% bind_cols(DF_4V) %>% mutate(Anomaly = "Yes")
# For normalities
DF_4V <- matr_normals %>% t() %>% c() %>% as.data.frame()
colnames(DF_4V) <- "AnalogVal"
DF_4T <- matr_normalT %>% t() %>% c() %>% as.POSIXct() %>% as.data.frame()
colnames(DF_4T) <- "StartDate"
DF_N <- DF_4T %>% bind_cols(DF_4V) %>% mutate(Anomaly = "No")
# join DF_A and DF_N...
DF_M4L <- DF_A %>% bind_rows(DF_N) %>% arrange(StartDate)
Let’s plot this

Indeed it make sense! This is what we want for our shinyapp…
Still some steps to do:
try to merge the MSE and interpret as anomaly level
Of course these manipulations with vectors and matrixes are possible. We can pack them to functions and so on. Before implementation, what about to simply convert the obtained result to another column that we can plot in parallel as a level of probability to have an anomaly in the data…
This will be less cumbersome and will give operators of the process possibility to have more clear understanding for interpretation…

ggplot(DF_F, aes(x = StartDate, y = AnalogVal, colour = AnomalyRating)) + geom_line() +
geom_linerange(data=DF_F,aes(ymin=AnalogVal, ymax=AnalogVal + AnomalyRating*50))

NA
Time to Shiny?
sasdfasfasfasfasdfasdfasf asdfasfasdfasdfasdfasdfasdf asdfasdfasdfasdfasfasdfasdf
And let’s not forget to switch off our cluster!
h2o.shutdown(prompt= FALSE)
Error in h2o.shutdown(prompt = FALSE) : There is no H2O instance running.
Conclusion
LS0tDQp0aXRsZTogIkxlY3R1cmUgMjcgLSBJbnRvIERlZXAgTGVhcm5pbmcgUDMiDQpvdXRwdXQ6DQogIGh0bWxfZG9jdW1lbnQ6IGRlZmF1bHQNCiAgaHRtbF9ub3RlYm9vazogZGVmYXVsdA0KICBwZGZfZG9jdW1lbnQ6IGRlZmF1bHQNCi0tLQ0KDQpUaGlzIGlzIGFuIFtSIE1hcmtkb3duXShodHRwOi8vcm1hcmtkb3duLnJzdHVkaW8uY29tKSBOb3RlYm9vaw0KDQoNCiMjIyBMZWN0dXJlIDI3IC0gWW91ciBwcm9qZWN0LiBEZWVwIExlYXJuaW5nIHdpdGggSDJPLiBQLjIgSW1wbGVtZW50IERlZXAgTGVhcm5pbmcgTW9kZWwgdG8gU2hpbnlBcHANCg0KRGVlcCBMZWFybmluZy4uLiBUaGlzIGxlY3R1cmUgaXMgZGVkaWNhdGVkIHRvIHRoZSBpbXBsZW1lbnRhdGlvbiBvZiBEZWVwIExlYXJuaW5nIG1vZGVscyBpbnRvIGZvciBvdXIgRGF0YQ0KDQojIyMjIFdvcmsgb3ZlcnZpZXcgKGZyb20gcHJldmlvdXMgbGVjdHVyZXMpDQoNCiogcmUtYXJyYW5naW5nIGRhdGEgYXMgbWF0cml4IC0gRE9ORQ0KKiBmaXR0aW5nIGRlZXAgbGVhcm5pbmcgbW9kZWxzIC0gRE9ORQ0KKiB0ZXN0aW5nIHRoZSBtb2RlbHMgLSBET05FDQoqIHNhdmluZyBtb2RlbHMgLSBET05FDQoqIGltcGxlbWVudGF0aW9uIGluIG91ciBTaGlueUFwcC4uLiAtIHRvIGJlIGNvdmVyZWQgaW4gdGhpcyBsZWN0dXJlDQoNCkluIG9yZGVyIHRvIGFjaGlldmUgdGhpcyBnb2FsIHdlIHdpbGwgaGF2ZSB0byBwZXJmb3JtIHNvbWUgc3RlcHM6DQoNCjEuIExlYXJuIGhvdyB0byB1c2Ugc2F2ZWQgbW9kZWwgaW4gUiB0byBtYWtlIHByZWRpY3Rpb25zDQoyLiBEZXZlbG9wIGFuZCBpbXBsZW1lbnQgYXBwIHVzYWdlIHN0cmF0ZWd5DQozLiBSZXR1cm4gdG8gb3VyIHRpbWUgc2VyaWVzIHRvIGlkZW50aWZ5IHRpbWUgb2YgYW5vbWFseQ0KDQojIyMjIFVzaW5nIHNhdmVkIGgybyBtb2RlbCBpbiBSDQoNCkluIHRoZSBwcmV2b3VzIGxlY3R1cmUgd2UgaGF2ZSBzYXZlZCBvdXIgRGVlcCBMZWFybmluZyBNb2RlbCBpbiB0aGUgd3d3IGZvbGRlcjoNCg0KYGBge3IsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9DQojZmlsZXMgd2l0aCBtb2RlbA0KZmlsZS5pbmZvKCJ3d3cvdG1wL25vcm1hbGl0eV9tb2RlbC5iaW4vRGVlcExlYXJuaW5nX21vZGVsX1JfMTUxMDU5NzQxMTY1Nl8xIikNCmBgYA0KT25lIG9mIHRoZSBHb2FscyBvZiB0aGlzIGxlY3R1cmUgd2lsbCBiZSB0byBzZWUgaG93IHRvIHVzZSB0aGlzIG1vZGVsIGluIFIgdG8gbWFrZSBwcmVkaWN0aW9ucw0KDQpGcm9tOiAoUiBoMm8gbG9hZCBhIHNhdmVkIG1vZGVsIGZyb20gZGlzayBpbiBNT0pPIG9yIFBPSk8gZm9ybWF0KVtodHRwczovL3N0YWNrb3ZlcmZsb3cuY29tL3F1ZXN0aW9ucy80NTMzNTY5Ny9yLWgyby1sb2FkLWEtc2F2ZWQtbW9kZWwtZnJvbS1kaXNrLWluLW1vam8tb3ItcG9qby1mb3JtYXRdDQoNCldlIGtub3cgdGhhdCB0d28gdHlwZXMgb2Ygc3RyYXRlZ3kgYXJlIHBvc3NpYmxlDQoNCjEuIHVzZSBhIGJpbmFyeSBtb2RlbCANCjIuIGV4cG9ydCBhIG1vZGVsIHRvIE1PSk8gKG9yIFBPSk8pIGZvcm0NCg0KIyMjIyBVc2luZyBCaW5hcnkgTW9kZWwgdG8gbWFrZSBwcmVkaWN0aW9ucw0KDQpXZSB3aWxsIGZpcnN0IHVzZSB0aGUgZmlyc3Qgb3B0aW9uIHRvIGFjdHVhbGx5IGtlZXAgSDJPIG9uIHRoZSBiYWNrZ3JvdW5kIHRvIG1ha2UgcHJlZGljdGlvbnMNCg0KYGBge3IsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9DQpsaWJyYXJ5KGgybykNCmxpYnJhcnkodGlkeXZlcnNlKQ0KbGlicmFyeShwbG90bHkpDQpgYGANCg0KSW5pdGlhbGl6aW5nIEgyTyBhbmQgbG9hZGluZyB0aGUgbW9kZWwNCg0KYGBge3IsIGV2YWw9VFJVRSwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRSwgaW5jbHVkZT1UUlVFfQ0KIyBpbml0aWFsaXplIGgybw0KaDJvLmluaXQoKQ0KIyBsb2FkIG1vZGVsDQpub3JtYWxpdHlfbW9kZWwgPC0gaDJvLmxvYWRNb2RlbCgid3d3L3RtcC9ub3JtYWxpdHlfbW9kZWwuYmluL0RlZXBMZWFybmluZ19tb2RlbF9SXzE1MTA1OTc0MTE2NTZfMSIpIA0KYGBgDQoNCldlIHdpbGwgYWxzbyBuZWVkIHRvIHF1aWNrbHkgY29uc3RydWN0IHRoZSBkYXRhc2V0IHdlIGNhbiBwcmVkaWN0IHRoZSByZXN1bHQNCg0KYGBge3IsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9DQoNCiMgPT09PT09PT09PT09PSBSRUFEIERBVEEgPT09PT09PT09PT09PT09PT0NCiMgUmVhZCBvdXIgc21hbGwgZGF0YSAuLi4gDQpERl9EYXRhX1JlY2VudCA8LSByZWFkUkRTKCJERl9EYXRhX1Byb2Nlc3NfUmVjZW50LmRhdGEiKSANCg0KREZfRXF1aXBtIDwtIHJlYWRfY3N2KCJERl9FcXVpcG1EYXRhLmNzdiIpDQojIGRhdGEgZnJhbWUgY29udGFpbmluZyBFdmVudCBOYW1lcw0KREZfRXZDb2RlIDwtIHJlYWRfY3N2KCJERl9FdkNvZGVEYXRhUHJvamVjdC5jc3YiKQ0KDQojIERhdGEgbWFuaXB1bGF0aW9uIGFuZCBzYXZpbmcgdG8gdGhlIERGX1RFTVANCkRGX1RFTVAgPC0gREZfRGF0YV9SZWNlbnQgJT4lIA0KICAjIGpvaW4gdG8gZGVjb2RlIGVxdWlwbWVudCBzZXJpYWwgbnVtYmVyDQogIGlubmVyX2pvaW4oREZfRXF1aXBtLCBieSA9ICJJREVxdWlwbWVudCIpICU+JSANCiAgIyBqb2luIHRvIGRlY29kZSBFdmVudCBDb2RlIG1lYW5pbmcNCiAgaW5uZXJfam9pbihERl9FdkNvZGUsIGJ5ID0gIkV2ZW50Q29kZSIpICU+JSANCiAgIyBzZWxlY3Qgb25seSBjb2x1bW4gbmVlZGVkDQogIHNlbGVjdChTdGFydERhdGUsIE5hbWUsIEFuYWxvZ1ZhbCwgRXZlbnRUZXh0KQ0KYGBgDQoNCg0KV2lsbCBiZSB1c2luZyBNYWNoaW5lICM0IGZvciB0aGF0DQoNCmBgYHtyfQ0KIyBpbXBvcnRpbmcgUiBmdW5jdGlvbiBmcm9tIGZpbGUNCnNvdXJjZSgidG9fbWF0cml4LlIiKQ0KDQpERl9NNCA8LSBERl9URU1QICU+JSANCiAgdG9fbWF0cml4KGZpbHRlcl9FdmVudCA9ICJUdWJpbmcgUHJvY2VzcywgcmVzaXN0YW5jZSBPaG0iLA0KICAgICAgICAgICAgZmlsdGVyX01hY2hpbmUgPSAiTWFjaGluZSAjNCIsbl9jb2xzID0gMTUwKQ0KYGBgDQoNCg0KIyMjIyBQcmVkaWN0aW5nIGFub21hbHkgb24gTWFjaGluZSA0DQoNCkluIG9yZGVyIHRvIHByZWRpY3Qgb24gYERGX000YCBkYXRhc2V0IHdlIG5lZWQgdG8gbWFrZSBpdCBIMk9GcmFtZSBjbGFzcy4gV2UgbmVlZCB0byBsb2FkIHRoaXMgb2JqZWN0IGludG8gSDJPIGVudmlyb25tZW50DQoNCmBgYHtyfQ0KIyBsb2FkIGRhdGFzZXQgaW50byBIMk8gZW52aXJvbm1lbnQNCnRlc3RfTTQgIDwtIGFzLmgybyh4ID0gREZfTTQsIGRlc3RpbmF0aW9uX2ZyYW1lID0gInRlc3RfTTQiKQ0KYGBgDQoNCg0KYGBge3J9DQojIHBsb3QgdGhlIGFub21hbGllcw0KaDJvLmFub21hbHkobm9ybWFsaXR5X21vZGVsLCB0ZXN0X000KSAlPiUgYXMuZGF0YS5mcmFtZSgpICU+JSBwbG90LnRzKHR5cGUgPSAicCIpDQpgYGANCg0KVGhpcyBicmluZ3MgdXMgdG8gdGhlIHBvaW50IG9mIGNyZWF0ZSBhICdjdXRvZmYnIHdoaWNoIEkgd2lsbCBzZXQgYXMgMC4yNS4gTVNFIHZhbHVlcyBzY29yZWQgYWJvdmUgdGhpcyB2YWx1ZSB3aWxsIGJlIGlkZW50aWZ5ZWQgYXMgYW5vbWFseS4gDQoNCmBgYHtyfQ0KIyBzYXZlIE1TRSB2YWx1ZXMNCm1zZV9vdXQgPC0gaDJvLmFub21hbHkobm9ybWFsaXR5X21vZGVsLCB0ZXN0X000KSAlPiUgYXMuZGF0YS5mcmFtZSgpDQojIGZpbHRlciBhbm9tYWxpZXMNCnJvd3NfYW5vbWFscyA8LSB3aGljaChtc2Vfb3V0ID4gMC4yNSkNCnJvd3Nfbm9ybWFscyA8LSB3aGljaChtc2Vfb3V0IDw9IDAuMjUpDQpgYGANCg0KVXNpbmcgdGhlc2Ugcm93cyB3ZSBjYW4gJ2xhYmVsJyBvciBzdWJzZXQgb3VyIGFub21hbG91cyBvYnNlcnZhdGlvbnMNCg0KYGBge3J9DQojIHN1YnNldCByb3dzIHdpdGggbWF0cmljZXMgdGhhdCBzaG93cyB0aGUgYW5vbWFsaWVzDQptYXRyX2Fub21hbHMgPC0gREZfTTRbcm93c19hbm9tYWxzLCBdDQoNCiMgc3Vic2V0IHJvd3Mgd2l0aCBtYXRyaWNlcyB0aGF0IGRvZXMgbm90IHNob3cgdGhlIGFub21hbGllcw0KbWF0cl9ub3JtYWxzIDwtIERGX000W3Jvd3Nfbm9ybWFscywgXSANCmBgYA0KDQpKdXN0IG91dCBvZiBjdXJpb3NpdHkgSSB3aWxsIHBsb3QgdGhlIGFub21hbGllcyB2YWx1ZXMgYXMgM0QgcGxvdA0KDQpgYGB7ciwgZXZhbD1GQUxTRSwgaW5jbHVkZT1UUlVFfQ0KcGxvdF9seSh6ID0gbWF0cl9hbm9tYWxzLCB0eXBlID0gInN1cmZhY2UiKQ0KYGBgDQohW0V4dHJhY3RlZCBvYnNlcnZhdGlvbnMgd2l0aCBhbm9tYWx5XVtpZDFdDQoNClRoaXMgaXMgd29uZGVyZnVsISBIaWdoIGFub21hbG91cyBzcGlrZXMgd2VyZSBkZXRlY3RlZCENCg0KQW5kIGFsc28gdGhvc2Ugd2l0aG91dA0KDQpgYGB7ciwgZXZhbD1GQUxTRSwgaW5jbHVkZT1UUlVFfQ0KcGxvdF9seSh6ID0gbWF0cl9ub3JtYWxzLCB0eXBlID0gInN1cmZhY2UiKQ0KYGBgDQoNClRoZXJlIGlzIG5vIHNwaWtlcyEgDQoNCiFbRGF0YSBwb2ludHMgd2l0aG91dCBhbm9tYWx5XVtpZDJdDQoNCiMjIyMgQ2F0Y2hpbmcgY29ycmVzcG9uZGluZyBkYXRlcyAmIHRpbWUNCg0KT25jZSB3ZSBsZWFybmVkIHRvIHByb2Nlc3Mgb3VyIGRhdGEgdGhyb3VnaCB0aGUgbW9kZWwsIHNwbGl0ICdmYXVsdHknIG9ic2VydmF0aW9ucyB3ZSB3aWxsIG5lZWQgdG8gY29tZSBiYWNrIHRvIHJlZmVyZW5jZSBvdXIgZmF1bHR5IG9ic2VydmF0aW9ucyBvdmVyIHRpbWUuIFBlcmhhcHMgdGhlcmUgYXJlIHdheXMgdG8gc29sdmUgdGhpcyBwcm9ibGVtIGluIGEgbW9yZSBlZmZpY2llbnQgd2F5LCBidXQgSSB3aWxsIGF0dGVtcHQgdG86DQoNCiogZXh0cmFjdCBjb3JyZXNwb25kaW5nIGRhdGUgdGltZQ0KKiBleHRyYWN0IG1hdHJpeGVzIHdpdGggZmF1bHRzIGZvciBib3RoIHZhbHVlcyBhbmQgdGltZQ0KKiBjb252ZXJ0IG1hdHJpY2VzIHRvIHZlY3RvcnMNCiogam9pbiBmYXVsdHkgb2JzZXJ2YXRpb25zIHdpdGggY29ycmVzcG9uZGluZyB0aW1lLCBhZGQgbmV3IGNvbHVtbiB3aXRoIGxhYmVsICIxIg0KKiBqb2luICdnb29kJyBvYnNlcnZhdGlvbnMgd2l0aCBjb3JyZXNwb25kaW5nIHRpbWUsIGFkZCBuZXcgY29sdW1uIHdpdGggbGFiZWwgIjAiDQoqIGNvbnN0cnVjdCB0aGUgZGF0YWZyYW1lcyBhbmQgYXJyYW5nZSB0aGVtIGJhY2sNCg0KVG8gZXh0cmFjdCB0aGUgdGltZS4uLiBJIGhhdmUgbW9kaWZpZWQgb3VyIGZ1bmN0aW9uIHRoYXQgZXh0cmFjdHMgYEFuYWxvZ1ZhbGAgdmFyaWFibGUgYW5kIHNhdmVkIHRoYXQgdG8gdGhlIFIgc2NyaXB0IGB0b19tYXRyaXhEVC5SYA0KDQpgYGB7cn0NCiMgaW1wb3J0aW5nIFIgZnVuY3Rpb24gZnJvbSBmaWxlDQpzb3VyY2UoInRvX21hdHJpeERULlIiKQ0KYGBgDQoNCkxldCdzIHVzZSB0aGlzIGZ1bmN0aW9uIHRvIGV4dHJhY3QgY29ycmVzcG9uZGluZyBEYXRlIGFuZCBUaW1lLi4uDQoNCmBgYHtyLCBldmFsPVRSVUUsIGluY2x1ZGU9VFJVRX0NCiMgcHJlcGFyZWQgbWF0cml4IGNvbnRhaW5pbmcgZGF0ZSBhbmQgdGltZSBlbGVtZW50cw0KREZfTTREVCA8LSB0b19tYXRyaXhEVChERl9URU1QLCANCiAgICAgICAgICAgICAgICAgICBmaWx0ZXJfRXZlbnQgPSAiVHViaW5nIFByb2Nlc3MsIHJlc2lzdGFuY2UgT2htIiwNCiAgICAgICAgICAgICAgICAgICBmaWx0ZXJfTWFjaGluZSA9ICJNYWNoaW5lICM0IiwgDQogICAgICAgICAgICAgICAgICAgbl9jb2xzID0gMTUwKQ0KYGBgDQoNCm5vdyBJIGNhbiBleHRyYWN0IGNvcnJlc3BvbmRpbmcgbWF0cml4ZXMNCg0KYGBge3J9DQojIHN1YnNldCByb3dzIHdpdGggbWF0cmljZXMgdGhhdCBzaG93cyB0aGUgYW5vbWFsaWVzDQptYXRyX2Fub21hbHMgPC0gREZfTTRbcm93c19hbm9tYWxzLCBdDQptYXRyX2Fub21hbFQgPC0gREZfTTREVFtyb3dzX2Fub21hbHMsIF0gI2NvcnJlc3BvbmRpbmcgdGltZSBkYXRlIGluZGV4DQoNCiMgc3Vic2V0IHJvd3Mgd2l0aCBtYXRyaWNlcyB0aGF0IGRvZXMgbm90IHNob3cgdGhlIGFub21hbGllcw0KbWF0cl9ub3JtYWxzIDwtIERGX000W3Jvd3Nfbm9ybWFscywgXSANCm1hdHJfbm9ybWFsVCA8LSBERl9NNERUW3Jvd3Nfbm9ybWFscywgXSAjY29ycmVzcG9uZGluZyB0aW1lIGRhdGUgaW5kZXgNCmBgYA0KDQpDb252ZXJ0IHRvIHZlY3RvcnMsIGpvaW4gYW5kIG11dGF0ZQ0KDQpgYGB7cn0NCiMgRm9yIGFub21hbGllcw0KREZfNFYgPC0gbWF0cl9hbm9tYWxzICU+JSB0KCkgJT4lIGMoKSAlPiUgYXMuZGF0YS5mcmFtZSgpIA0KY29sbmFtZXMoREZfNFYpIDwtICJBbmFsb2dWYWwiDQpERl80VCA8LSBtYXRyX2Fub21hbFQgJT4lIHQoKSAlPiUgYygpICU+JSBhcy5QT1NJWGN0KCkgJT4lICBhcy5kYXRhLmZyYW1lKCkgDQpjb2xuYW1lcyhERl80VCkgPC0gIlN0YXJ0RGF0ZSINCkRGX0EgPC0gREZfNFQgJT4lIGJpbmRfY29scyhERl80VikgJT4lIG11dGF0ZShBbm9tYWx5ID0gIlllcyIpDQoNCiMgRm9yIG5vcm1hbGl0aWVzDQpERl80ViA8LSBtYXRyX25vcm1hbHMgJT4lIHQoKSAlPiUgYygpICU+JSBhcy5kYXRhLmZyYW1lKCkgDQpjb2xuYW1lcyhERl80VikgPC0gIkFuYWxvZ1ZhbCINCkRGXzRUIDwtIG1hdHJfbm9ybWFsVCAlPiUgdCgpICU+JSBjKCkgJT4lIGFzLlBPU0lYY3QoKSAlPiUgYXMuZGF0YS5mcmFtZSgpDQpjb2xuYW1lcyhERl80VCkgPC0gIlN0YXJ0RGF0ZSINCkRGX04gPC0gREZfNFQgJT4lIGJpbmRfY29scyhERl80VikgJT4lIG11dGF0ZShBbm9tYWx5ID0gIk5vIikNCg0KIyBqb2luIERGX0EgYW5kIERGX04uLi4NCkRGX000TCA8LSBERl9BICU+JSBiaW5kX3Jvd3MoREZfTikgJT4lIGFycmFuZ2UoU3RhcnREYXRlKQ0KDQoNCmBgYA0KDQpMZXQncyBwbG90IHRoaXMNCg0KYGBge3J9DQpnZ3Bsb3QoREZfTTRMLCBhZXMoeCA9IFN0YXJ0RGF0ZSwgeSA9IEFuYWxvZ1ZhbCwgY29sID0gYXMuZmFjdG9yKEFub21hbHkpKSkgKyBnZW9tX2xpbmUoKQ0KYGBgDQoNCkluZGVlZCBpdCBtYWtlIHNlbnNlISBUaGlzIGlzIHdoYXQgd2Ugd2FudCBmb3Igb3VyIHNoaW55YXBwLi4uDQoNClN0aWxsIHNvbWUgc3RlcHMgdG8gZG86IA0KDQojIyMjIHRyeSB0byBtZXJnZSB0aGUgTVNFIGFuZCBpbnRlcnByZXQgYXMgYW5vbWFseSBsZXZlbA0KDQpPZiBjb3Vyc2UgdGhlc2UgbWFuaXB1bGF0aW9ucyB3aXRoIHZlY3RvcnMgYW5kIG1hdHJpeGVzIGFyZSBwb3NzaWJsZS4gV2UgY2FuIHBhY2sgdGhlbSB0byBmdW5jdGlvbnMgYW5kIHNvIG9uLiBCZWZvcmUgaW1wbGVtZW50YXRpb24sIHdoYXQgYWJvdXQgdG8gc2ltcGx5IGNvbnZlcnQgdGhlIG9idGFpbmVkIHJlc3VsdCB0byBhbm90aGVyIGNvbHVtbiB0aGF0IHdlIGNhbiBwbG90IGluIHBhcmFsbGVsIGFzIGEgbGV2ZWwgb2YgcHJvYmFiaWxpdHkgdG8gaGF2ZSBhbiBhbm9tYWx5IGluIHRoZSBkYXRhLi4uDQoNClRoaXMgd2lsbCBiZSBsZXNzIGN1bWJlcnNvbWUgYW5kIHdpbGwgZ2l2ZSBvcGVyYXRvcnMgb2YgdGhlIHByb2Nlc3MgcG9zc2liaWxpdHkgdG8gaGF2ZSBtb3JlIGNsZWFyIHVuZGVyc3RhbmRpbmcgZm9yIGludGVycHJldGF0aW9uLi4uDQoNCmBgYHtyfQ0KIyBtc2Vfb3V0IGlzIGEgZGF0YWZyYW1lIGNvbnRhaW5pbmcgTVNFIGVycm9yIGZvciBlYWNoIHJvdyBvZiB0aGUgbWF0cml4LCB3ZSByZXBsaWNhdGUgdGhpcy4uLiAxNTAgdGltZXMuLi4NCkRGX000bXNlIDwtIGRvLmNhbGwoY2JpbmQsIHJlcGxpY2F0ZSgxNTAsIGFzLm1hdHJpeChtc2Vfb3V0KSwgc2ltcGxpZnk9RkFMU0UpKQ0KDQojIGdvaW5nIHRvIGNvbWJpbmUgdGhpcyByZXN1bHQgd2l0aCBvdXIgb2JqZWN0cyBERl9NNCBhbmQgREZfTTREVA0KREZfViA8LSBERl9NNCAlPiUgdCgpICU+JSBjKCkgJT4lIGFzLmRhdGEuZnJhbWUoKSANCmNvbG5hbWVzKERGX1YpIDwtICJBbmFsb2dWYWwiDQpERl9UIDwtIERGX000RFQgJT4lIHQoKSAlPiUgYygpICU+JSBhcy5QT1NJWGN0KCkgJT4lICBhcy5kYXRhLmZyYW1lKCkgDQpjb2xuYW1lcyhERl9UKSA8LSAiU3RhcnREYXRlIg0KREZfQSA8LSBERl9NNG1zZSAlPiUgdCgpICU+JSBjKCkgJT4lIGFzLmRhdGEuZnJhbWUoKSANCmNvbG5hbWVzKERGX0EpIDwtICJBbm9tYWx5UmF0aW5nIg0KDQpERl9GIDwtIERGX1YgJT4lIGJpbmRfY29scyhERl9ULCBERl9BKQ0KDQpsaWJyYXJ5KFJDb2xvckJyZXdlcikNCm15cGFsZXR0ZTwtYnJld2VyLnBhbCgzLCJQYWlyZWQiKQ0KDQpteXNwZWNpYWwgPC0gYygiIzkxY2Y2MCIsICIjZmZmZmJmIiwgIiNmYzhkNTkiKQ0KDQpnZ3Bsb3QoREZfRiwgYWVzKHggPSBTdGFydERhdGUsIHkgPSBBbmFsb2dWYWwsIGNvbG91ciA9IEFub21hbHlSYXRpbmcpKSArIGdlb21fbGluZSgpICsgc2NhbGVfY29sb3VyX2dyYWRpZW50bihjb2xvdXJzPW15c3BlY2lhbCkNCg0KYGBgDQoNCg0KYGBge3J9DQpnZ3Bsb3QoREZfRiwgYWVzKHggPSBTdGFydERhdGUsIHkgPSBBbmFsb2dWYWwsIGNvbG91ciA9IEFub21hbHlSYXRpbmcpKSArIGdlb21fbGluZSgpICsgDQogIGdlb21fbGluZXJhbmdlKGRhdGE9REZfRixhZXMoeW1pbj1BbmFsb2dWYWwsIHltYXg9QW5hbG9nVmFsICsgQW5vbWFseVJhdGluZyo1MCkpDQogIA0KDQoNCmBgYA0KDQojIyMjIFRpbWUgdG8gU2hpbnk/DQoNCnNhc2RmYXNmYXNmYXNmYXNkZmFzZGZhc2YNCmFzZGZhc2Zhc2RmYXNkZmFzZGZhc2RmYXNkZg0KYXNkZmFzZGZhc2RmYXNkZmFzZmFzZGZhc2RmDQoNCg0KQW5kIGxldCdzIG5vdCBmb3JnZXQgdG8gc3dpdGNoIG9mZiBvdXIgY2x1c3RlciENCmBgYHtyLCBldmFsPVRSVUUsIGluY2x1ZGU9VFJVRX0NCmgyby5zaHV0ZG93bihwcm9tcHQ9IEZBTFNFKQ0KDQpgYGANCg0KIyMjIyBDb25jbHVzaW9uDQoNCg0KDQoNCiMjIyMgTmV4dCBzdGVwDQoNCi4uLg0KDQoNCiMjIyMgdXNlZCByZWZlcmVuY2VzDQoNCmV4YW1wbGUgZnJvbTogKGh0dHBzOi8vZHpvbmUuY29tL2FydGljbGVzL2Fub21hbHktZGV0ZWN0aW9uLXdpdGgtZGVlcC1sZWFybmluZy1pbi1yLXdpdGgtaDJvKVtodHRwczovL2R6b25lLmNvbS9hcnRpY2xlcy9hbm9tYWx5LWRldGVjdGlvbi13aXRoLWRlZXAtbGVhcm5pbmctaW4tci13aXRoLWgyb10NCg0KTW9yZSByZWFkaW5nOiAoaHR0cHM6Ly9kem9uZS5jb20vYXJ0aWNsZXMvdGhlLWJhc2ljcy1vZi1kZWVwLWxlYXJuaW5nLWhvdy10by1hcHBseS1pdC10by1wcmU/ZnJvbXJlbD10cnVlKVtodHRwczovL2R6b25lLmNvbS9hcnRpY2xlcy90aGUtYmFzaWNzLW9mLWRlZXAtbGVhcm5pbmctaG93LXRvLWFwcGx5LWl0LXRvLXByZT9mcm9tcmVsPXRydWVdDQoNClRoZSBwaHlsb3NvcGh5IHRoYXQgd2FzIGFscmVhZHkgZXhwbGFpbmVkIGluIHRoZSBjb3Vyc2UgKEkgZGlzY292ZXJlZCB0aGF0IGxhdGVyLi4uOikgOiAoaHR0cHM6Ly9kem9uZS5jb20vYXJ0aWNsZXMvZGl2ZS1kZWVwLWludG8tZGVlcC1sZWFybmluZy11c2luZy1oMm8tMSlbaHR0cHM6Ly9kem9uZS5jb20vYXJ0aWNsZXMvZGl2ZS1kZWVwLWludG8tZGVlcC1sZWFybmluZy11c2luZy1oMm8tMV0NCg0KQW5kOiAoaHR0cHM6Ly9zaGlyaW5nLmdpdGh1Yi5pby9tYWNoaW5lX2xlYXJuaW5nLzIwMTcvMDUvMDEvZnJhdWQpW2h0dHBzOi8vc2hpcmluZy5naXRodWIuaW8vbWFjaGluZV9sZWFybmluZy8yMDE3LzA1LzAxL2ZyYXVkXQ0KDQpwYXBlcjogKGh0dHBzOi8vYXJ4aXYub3JnL2Ficy8xNzAxLjAxODg3KVtodHRwczovL2FyeGl2Lm9yZy9hYnMvMTcwMS4wMTg4N10NCkluIHRoaXMgbGVjdHVyZSB3ZSB3b3VsZCBleHBsb3JlIHRoZSAndGVjaG5vbG9neScgb24gdGhlIHNhbXBsZSBhbmQgdHJ5IHRvIGRvIHRoaXMgaW4gMTAgbWluIGxlY3R1cmUhDQoNCg0KDQpbaWQxXTogcGxvdHMvTTRfYW5vbWFseS5wbmcgIkV4dHJhY3RlZCBvYnNlcnZhdGlvbnMgd2l0aCBhbm9tYWx5Ig0KW2lkMl06IHBsb3RzL000X25vX2Fub21hbHkucG5nICJPYnNlcnZhdGlvbnMgd2l0aG91dCBhbm9tYWx5Ig0KDQoNCg0K